Recent literature has described a reconfigurable network topology amenable to evolving data workloads such as stream processing. For example, K. J. Barker et al., On the Feasibility of Optical Circuit Switching for High Performance Computing Systems, Proceedings of the ACM/IEEE SC 2005 Conference, the disclosure of which is incorporated by reference herein, describes a network topology that combines the flexibility and power savings of Optical Circuit Switches, with an aggregation scheme whereby several nodes are connected to one of several Electrical Packet Switches, and each Electrical Packet Switch is connected to the Optical Circuit Switch. The Optical Circuit Switch can be dynamically reconfigured as bandwidth needs evolve to connect one or more ports of each Electrical Packet Switch to one or more ports on other Electrical Packet Switches. In this way, data can be sent from a node, aggregated with traffic from other nodes at their packet switch, routed through the Optical Circuit Switch which is dynamically configured to connect packet switches together that need the highest bandwidth, and routed through the target Electrical Packet Switch and then demultiplexed to target nodes. However, the technique described in the Barker paper for routing through the reconfigurable network to maximize network utilization is dependent on the development of new communication protocols—there is no support for existing network protocols, such as Transmission Control Protocol/Internet Protocol (TCP/IP).
To route protocols such as TCP/IP through the reconfigurable network while attempting to maximize the network utilization, there are a few well-known options. For example, standard network routing on Layer 2 involves the use of a default LAN and setup spanning tree group on that LAN to remove loops. The source node can send data to the destination on the default LAN. The primary disadvantage of this approach is that some links will be disabled by the spanning tree algorithm resulting in significantly reduced network link utilization. Additionally, the spanning tree protocol may take seconds to converge for each topology reconfiguration.
Another approach involves source routing, in which the whole route is stored in each packet header and a switch will route the packet according to the path specified in the header. The primary disadvantage of this approach is that an additional protocol needs to be implemented on the packet switch and nodes to do source routing. The route through the network has to be communicated to the origin so that it can be included in the packet. Additionally, packet size increases as network size and complexity increases.
Virtual circuit routing may be used in which, after each topology reconfiguration, virtual circuits are setup between chassis for routing. The disadvantage of this approach is that it requires dynamic virtual circuit assignment at the packet switch for each route after each reconfiguration. This requires that the packet switch have virtual circuit support at external ports; most switches do not have this capability.
Virtual local area networks (VLANs) have been employed to create a logical separation on a shared physical local area network VLANs have also been used to provide a primary and backup network for fail-over situations. For example, U.S. Pat. No. 7,231,430, the disclosure of which is incorporated by reference herein, teaches the use of reconfigurable VLANs to partition a computer cluster for different virtual services accessed by an external network.